Selecting Diverse Features via Spectral Regularization
نویسندگان
چکیده
We study the problem of diverse feature selection in linear regression: selecting a small subset of diverse features that can predict a given objective. Diversity is useful for several reasons such as interpretability, robustness to noise, etc. We propose several spectral regularizers that capture a notion of diversity of features and show that these are all submodular set functions. These regularizers, when added to the objective function for linear regression, result in approximately submodular functions, which can then be maximized by efficient greedy and local search algorithms, with provable guarantees. We compare our algorithms to traditional greedy and `1-regularization schemes and show that we obtain a more diverse set of features that result in the regression problem being stable under perturbations.
منابع مشابه
Automatic Feature Selection via Weighted Kernels and Regularization
Selecting important features in non-linear kernel spaces is a difficult challenge in both classification and regression problems. We propose to achieve feature selection by optimizing a simple criterion: a feature-regularized loss function. Features within the kernel are weighted, and a lasso penalty is placed on these weights to encourage sparsity. We minimize this feature-regularized loss fun...
متن کاملSelecting the Corner in the -Curve Approach to Tikhonov Regularization
The performance of two methods for selecting the corner in the -curve approach to Tikhonov regularization is evaluated via computer simulation. These methods are selecting the corner as the point of maximum curvature in the -curve, and selecting it as the point where the product of abcissa and ordinate is a minimum. It is shown that both these methods resulted in significantly better regulariza...
متن کاملImproving of Feature Selection in Speech Emotion Recognition Based-on Hybrid Evolutionary Algorithms
One of the important issues in speech emotion recognizing is selecting of appropriate feature sets in order to improve the detection rate and classification accuracy. In last studies researchers tried to select the appropriate features for classification by using the selecting and reducing the space of features methods, such as the Fisher and PCA. In this research, a hybrid evolutionary algorit...
متن کاملc-LASSO and its dual for sparse signal estimation from array data
We treat the estimation of a sparse set of sources emitting plane waves observed by a sensor array as a complex-valued LASSO (c–LASSO) problem where the usual l1-norm constraint is replaced by the l1-norm of a matrix D times the solution vector. When the sparsity order is given, algorithmically selecting a suitable value for the c–LASSO regularization parameter remains a challenging task. The c...
متن کاملHyperspectral Image Classification Based on the Fusion of the Features Generated by Sparse Representation Methods, Linear and Non-linear Transformations
The ability of recording the high resolution spectral signature of earth surface would be the most important feature of hyperspectral sensors. On the other hand, classification of hyperspectral imagery is known as one of the methods to extracting information from these remote sensing data sources. Despite the high potential of hyperspectral images in the information content point of view, there...
متن کامل